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Abstract 


Single-look  identification  procedure  using  one  input  at  a  time  has  been  the  principal  mode  of 
analysis  used  in  the  target  recognition  community  for  much  of  the  past  two  decades.  But  it  has 
been  realized  that  the  single-look  approach  can  only  achieve  modest  correct  identification 
performance  and  this  is  not  adequate  for  many  target  identification  applications.  Furthermore,  in 
order  for  the  single-look  procedure  to  perform  well,  good  target  data  quality  is  required.  In  this 
report,  a  multi-look  method  known  as  score-level  fusion  is  investigated.  It  permits  a  significant 
improvement  in  the  identification  performance.  Moreover,  it  does  not  require  good  quality  target 
signature  data;  multiple  signature  samples  of  marginal  quality  may  be  used  instead.  Results  from 
analysis  using  measured  radar  target  data  have  shown  quantitatively  that  the  correct  identification 
rate  can  be  improved  very  dramatically.  Furthermore,  the  score-level  fusion  method  is  very 
efficient;  the  identification  performance  increases  exponentially  with  an  increase  in  the  number  of 
samples  used  in  the  multi-look  sequence.  A  qualitative  characterization  of  the  score-level  fusion 
method  based  on  the  principle  of  averaging  is  given.  It  provides  an  intuitive  understanding  of  this 
fusion  process  and  how  improvement  in  identification  accuracy  is  achieved.  The  identification 
results  obtained  in  this  study  suggest  that  quantity  can  be  used  to  replace  quality  of  the  data  to 
improve  target  identification  accuracy.  This  has  an  interesting  practical  implication.  With  the 
advent  of  sensor  technologies,  large  quantities  of  data  of  marginal  quality  can  be  captured 
routinely.  This  “quantity  over  quality”  approach  could  maximize  the  exploitation  of  available  data 
to  provide  reliable  and  robust  identification.  Thus,  the  score-level  fusion  method  could  have  the 
potential  of  being  developed  into  a  disruptive  technology  for  target  identification  and  pattern 
recognition  problems. 


Resume 


Pendant  la  plus  grande  partie  des  deux  dernieres  decennies,  la  communaute  de  la  reconnaissance 
des  objectifs  a  principalement  employe  comme  methode  d' analyse  la  procedure  d’ identification 
monovisee,  laquelle  exploite  les  donnees  obtenues  qu’avec  un  un  seul  capteur.  Toutefois,  force 
est  de  constater  que  le  rendement  de  V identification  correcte  d’objectifs  par  l’approche 
monovisee  est  modeste  et,  qu’en  outre,  celle-ci  n'est  pas  adaptee  a  plusieurs  applications  de 
F identification  d’objectifs.  Qui  plus  est,  la  procedure  monovisee  ne  donne  de  resultats  corrects 
que  si  on  1’ applique  qu’a  des  donnees  de  bonne  qualite  pour  les  objectifs.  Dans  le  present  rapport, 
nous  presentons  notre  etude  d’une  methode  multivisee  nettement  superieure  sur  le  plan  de 
F identification  :  la  fusion  au  niveau  des  scores.  En  outre,  pour  bien  fonctionner,  cette  methode 
n’exige  pas  que  les  donnees  de  signature  d’objectifs  soient  de  bonne  qualite,  elle  traite  plutot  un 
grand  nombre  d’echantillons  de  qualite  mediocre.  Les  resultats  d’une  analyse  realisee  avec  les 
donnees  d’un  objectif,  obtenues  par  radar  ont  quantitativement  montre  que  la  methode  de  fusion 
au  niveau  des  scores  pouvait  grandement  ameliorer  le  taux  d’ identifications  correctes.  Qui  plus 
est,  elle  est  tres  efficace  et  son  rendement  augmente  exponentiellement  avec  le  nombre 
d’echantillons  multivisees  utilises.  Nous  caracterisons  qualitativement  cette  methode  qui  repose 
sur  le  principe  de  la  moyenne.  Cette  caracterisation  apporte  une  comprehension  intuitive  du 
precede  de  fusion  et  de  la  fa5on  dont  il  ameliore  la  precision  de  l’identification.  Les  resultats 
d’ identification  obtenus  dans  la  presente  etude  suggerent  que  Ton  puisse  augmenter  la  precision 
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d' identification  des  objectifs  en  accroissant  la  quantite  de  donnees  plutot  qu'cn  en  ameliorant  la 
qualite.  Notre  travail  a  des  consequences  pratiques  interessantes.  Avec  l’apparition  de  nouvelles 
technologies  de  capteurs,  on  pourrait  obtenir  couramment  de  grandes  quantites  de  donnees  de 
qualite  mediocre.  Cette  approche  de  «  la  quantite  plutot  que  la  qualite  »  permettrait  de  maximiser 
1’ exploitation  des  donnees  disponibles  et  nous  fournir  des  identifications  fiables  et  solides.  Ainsi, 
la  methode  de  fusion  au  niveau  des  scores  pourrait  constituer  une  technologie  de  rupture  pour  les 
problemes  d’identification  des  objectifs  et  de  reconnaissance  des  formes. 


n 


DRDC  Ottawa  TM  2008-251 


Executive  summary 


Accurate  target  identification  using  multi-look  fusion  of  low 
quality  target  signatures: 

S.  Wong;  DRDC  Ottawa  TM  2008-251 ;  Defence  R&D  Canada  -  Ottawa; 

December  2008. 

Introduction  or  background:  Identification  algorithms  are  of  practical  interest  for  various 
military  and  security  applications.  These  include  Combat  Identification,  Aided  Target 
Recognition,  Low  Probability  of  Intercept  radar  identification  and  human  facial  recognition. 

Much  of  the  effort  in  the  target  recognition  community  so  far  has  been  devoted  to  achieving  high 
identification  accuracy  using  a  single  target  signature  as  input  to  the  identification  process.  It  is 
found  that  the  identification  accuracy  that  can  be  achieved  is  modest  and  is  dependent  on  the 
quality  of  the  target  signatures  used.  With  sufficiently  good  signature  quality,  correct 
identification  rate  in  the  high  eighties  and  low  nineties  percentage  can  be  achieved  readily.  But  in 
some  applications,  for  example  Combat  Identification,  identification  accuracy  better  than  99%  is 
sought.  To  improve  on  the  accuracy,  multi-look  methods  have  been  investigated,  in  which  a 
number  of  signatures  are  used  collectively  to  determine  the  identification  outcome.  Although  the 
multi-look  approach  can  improve  the  correct  identification  rate  considerably,  many  commonly 
used  multi-look  procedures  require  good  quality  signatures  since  they  are  based  on  examining  the 
collective  identification  evidence  from  a  group  of  single-look  samples.  Hence,  data  quality  still 
remains  an  important  issue. 

In  this  report,  a  different  multi-look  identification  approach  is  examined.  This  method  is  known 
as  the  score-level  fusion  method;  it  fuses  information  from  multiple  input  samples  in  making  an 
identification  decision.  Results  from  this  study  have  shown  that  the  score-level  fusion  approach 
does  not  require  good  quality  target  signature  data  to  achieve  highly  accurate  identification; 
multiple  samples  of  marginal  quality  may  be  used  instead.  No  study  has  yet  been  reported  in  the 
open  literature  in  examining  the  capability  of  an  identification  procedure  that  can  make  use  of  low 
quality  target  signatures,  but  produce  highly  accurate  identification. 

The  superior  performance  of  the  score-level  fusion  method  is  illustrated  in  a  comparative  analysis 
against  the  well-known  majority-vote  method.  The  quality  of  the  target  signature  data  and  the 
multi-look  sample  size  are  two  major  parameters  that  are  used  in  evaluating  the  performance 
between  these  two  multi-look  methods. 

Results:  The  majority-vote  method  is  a  well-established  and  commonly  used  algorithm  for  multi¬ 
look  identification  to  improve  identification  accuracy.  It  produces  an  identification  decision  based 
on  a  particular  target  type  appearing  the  most  frequently  from  a  series  of  single-look 
identification  outcomes.  However,  its  performance  is  highly  dependent  on  the  quality  of  the 
signatures.  Using  relatively  good  quality  data  such  that  the  single-look  correct  identification  rate 
is  significantly  above  50%,  the  majority-vote  method  can  provide  notable  improvement  to  the 
identification  performance.  But  for  data  of  marginal  quality,  the  majority-vote  method  does  not 
provide  any  improvement  at  all.  For  example,  when  the  signature  quality  is  at  such  poor  level  that 
the  single-look  correct  identification  rate  is  less  than  50%,  the  identification  performance  is 
actually  worse  by  applying  the  majority-vote  method.  In  contrast,  the  score -level  fusion  method  is 
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found  to  be  capable  of  providing  a  significant  improvement  to  the  identification  performance 
irrespective  of  the  quality  of  the  target  signature  data.  It  achieves  this  by  taking  a  different 
approach  to  multi-look  identification.  Instead  of  computing  an  identification  decision  from  each 
input  signature,  the  score-level  fusion  computes  a  series  of  “matching  scores”  from  multiple  input 
signatures.  These  matching  scores  are  then  combined  to  generate  a  single  fused  score  for  each  of 
the  target  classes  that  are  stored  in  the  classifier.  An  identification  decision  is  then  made  based  on 
the  fused  scores.  Using  data  of  relatively  poor  quality,  the  score-level  fusion  has  shown  that  it  is 
capable  of  improving  the  correct  identification  rate  from  37%  (in  single-look)  to  over  97%  using 
a  20-look  sequence.  Moreover,  the  correct  identification  rate  increases  exponentially  with  an 
increase  in  the  number  of  multi-look  samples  used.  This  represents  a  very  effective  means  for 
improving  the  identification  performance.  In  brief,  what  is  original  and  innovative  in  this  study  is 
the  demonstration  of  a  process  that  enables  significant  improvement  in  target  identification 
accuracy  using  multiple  samples  of  marginal  quality  signatures. 

Significance:  Results  from  the  score-level  fusion  method  indicate  that  quantity  can  be  used  to 
replace  quality  of  the  data  to  improve  the  identification  accuracy.  This  could  have  a  profound 
implication  for  practical  applications.  With  the  advent  of  sensor  technologies  and  deployment  of 
multiple  sensor  platforms  for  surveillance,  large  quantities  of  data  of  marginal  quality  can  be 
captured  readily  for  intelligence  exploitation;  whereas,  collecting  quality  data  still  presents  a 
technical  challenge  and  could  also  be  more  costly.  The  ability  to  maximize  the  exploitation  of  a 
large  number  of  available  data  to  improve  on  the  identification  performance  could  have 
significant  impact  in  intelligence  analysis.  This  “quantity  over  quality”  approach  could  have  the 
potential  of  being  a  disruptive  technology  for  Defence  and  Security  S&T  applications,  in 
particular,  in  the  area  of  ISR.  It  can  also  be  applied  to  other  pattern  recognition  problems  in 
general,  for  example,  human  biometrics  such  as  face  recognition.  Thus,  a  special  property  of  the 
score -level  fusion  method  has  been  uncovered;  that  is,  poor  quality  signature  data  can  be  of 
practical  value  and  can  be  exploited.  This  study  represents  a  major  step  forward  in  target 
identification  algorithm  development. 

Future  plans:  So  far,  the  score-level  fusion  method  has  been  applied  only  to  target  signature 
identification  analysis  in  the  radar  domain  in  our  research  activities.  In  this  report,  signatures  of 
ground  targets  in  the  X-band  (10  GHz)  are  analyzed  and  exceptionally  good  results  have  been 
achieved.  In  earlier  work,  in-flight  aircraft  signatures  in  the  Ku-band  (16  GHz)  were  examined. 
Excellent  identification  results  have  also  been  obtained.  Investigation  will  be  applied  to  the 
human  facial  recognition  problem.  A  research  project  “Video-based  facial  recognition  algorithm 
and  system  demonstration”  is  currently  in  progress  to  assess  the  score-level  fusion  method  for 
video  image  data. 

Furthermore,  there  is  very  little  theoretical  work  on  the  score-level  fusion  method  being  reported 
in  the  literature.  This  presents  a  challenge  and  an  opportunity  to  give  the  score-level  fusion 
method  a  more  precise  mathematical  description,  providing  a  rigorous  theoretical  foundation. 
Further  basic  research  would  be  desirable  and  of  strategic  value.  A  better  conceptual 
understanding  of  the  fundamentals  could  have  significant  impact  in  opening  up  new  applications 
to  other  pattern  recognition  problems,  particularly  in  the  Defence  and  Security  S&T  area. 


IV 


DRDC  Ottawa  TM  2008-251 


Sommaire 


Accurate  target  identification  using  multi-look  fusion  of  low 
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Introduction  ou  contexte:  Les  algorithmes  d’ identification  sont  d’un  interet  pratique  aux  plans 
militaire  et  securitaire,  pensons  notamment  a  1' identification  pendant  le  combat,  a  la 
reconnaissance  assistee  d'objectifs,  a  1’ identification  par  radar  a  faible  probabilite  d'interception  et 
a  la  reconnaissance  des  visages.  Jusqu’a  maintenant,  la  communaute  de  reconnaissance  des 
objectifs  a  consacre  une  grande  partie  de  ses  travaux  a  accroitre  la  precision  de  1’ identification,  a 
partir  d'une  seule  signature  d’objectif.  Or,  l’on  a  decouvert  que  la  precision  d’ identification  que 
Ton  pouvait  obtenir  etait  modeste  et  variait  en  fonction  de  la  qualite  des  signatures  d’objectifs 
employees.  Ainsi,  si  l’on  dispose  de  signatures  dont  la  qualite  est  suffisamment  bonne,  on  peut 
obtenir  d’emblee  un  taux  d’ identifications  correctes  autour  de  90  %.  Toutefois,  dans  certaines 
applications,  1’ identification  au  combat  notamment,  cette  precision  doit  depasser  99  %.  Pour  la 
rehausser,  on  a  realise  des  recherches  sur  des  methodes  multivisees  qui  etablissent  une 
identification  par  l’emploi  collectif  de  plusieurs  signatures.  Les  methodes  multivisees  pourraient 
grandement  ameliorer  le  taux  d’ identifications  correctes.  Toutefois,  plusieurs  d'entre  elles  sont 
fondees  sur  Texamen  collectif  de  preuves  d’ identification  a  partir  d'un  ensemble  d’ echantillons 
monovisees  et  done  necessitent  des  signatures  de  bonne  qualite.  Le  probleme  de  la  qualite  des 
donnees  reste  done  un  enjeu  important. 

Dans  le  present  rapport,  nous  presentons  notre  examen  d’une  methode  differente  d’ identification 
multivisee  :  la  fusion  au  niveau  des  scores  qui  arrive  a  une  decision  d’ identification  par 
l’integration  des  informations  de  plusieurs  echantillons  a  T  entree.  Nous  avons  montre  que,  pour 
arriver  a  une  identification  tres  precise,  la  fusion  au  niveau  des  scores  n’exigeait  pas  de  donnees 
de  signature  de  bonne  qualite,  mais  se  contentait  de  plusieurs  echantillons  de  qualite  mediocre. 
Dans  la  litterature  publiee,  nous  n’avons  pas  trouve  d’etude  de  la  capacite  d’une  procedure 
d’ identification  pouvant  produire  des  identifications  de  haute  qualite,  a  partir  de  signatures 
d'objectifs  de  basse  qualite. 

Nous  montrons  le  rendement  superieur  de  la  methode  de  fusion  des  scores,  grace  a  une  analyse 
comparative  avec  la  methode  bien  connue  du  «  vote  majoritaire  ».  La  qualite  des  donnees  de 
signature  et  la  taille  des  echantillons  multivisees  sont  deux  parametres  importants  utilises  pour 
comparer  les  rendements  de  ces  deux  methodes  multivisees. 


Resultats:  La  methode  du  vote  majoritaire  est  un  algorithme  bien  etabli  et  couramment  employe 
pour  ameliorer  la  precision  de  1’ identification  a  partir  d’ informations  multivisees.  Elle  donne  une 
decision  sur  1’ identification  en  «  elisant  »  le  type  particular  d’objectif  le  plus  frequemment 
retrouve  dans  une  serie  de  resultats  d’ identifications  monovisees.  Ainsi,  son  rendement  depend 
fortement  de  la  qualite  des  signatures.  Si  Ton  dispose  de  donnees  relativement  bonnes  dont  la 
proportion  de  1’ identification  monovisee  correcte  est  bien  au-dela  de  50  %,  cette  methode 
ameliorera  considerablement  le  rendement  d’ identification.  Si,  par  contre,  les  donnees  sont 
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mediocres,  elle  n’  apportera  aucune  amelioration.  Par  exemple,  si  la  qualite  des  signatures  est 
mauvaise  au  point  que  le  taux  d’identifications  monovisees  correctes  est  inferieur  a  50  %,  la 
methode  du  vote  majoritaire  donnera  un  rendement  pire.  En  revanche,  nous  avons  trouve  que  la 
methode  de  fusion  au  niveau  des  scores  ameliorait  grandement  le  rendement  d' identification, 
quelle  que  soit  la  qualite  des  donnees  de  signature  des  objectifs.  Cette  independance  par  rapport  a 
la  qualite  provient  du  traitement  different  de  1’  identification  multivisee  :  plutot  que  de  calculer  la 
decision  d’identification  a  partir  de  chaque  signature  a  l’entree,  elle  calcule  une  serie  de  «  scores 
apparies  »  a  partir  de  multiples  signatures  a  l’entree.  Ces  scores  apparies  sont  ensuite  combines 
pour  produire  un  unique  score  fusionne  pour  chaque  classe  d’ objectifs  conservee  dans  le 
classificateur.  La  decision  est  ensuite  prise  a  partir  des  scores  fusionnes.  Appliquee  sur  des 
donnees  plutot  mediocres,  la  methode  de  fusion  au  niveau  des  scores  a  pu  ameliorer  le  taux 
d' identifications  correctes  de  37  %  a  plus  de  97  %  en  passant  de  donnees  monovisees  a  sequence 
de  vingt  «  visees  ».  Qui  plus  est,  le  taux  d’identifications  correctes  augmente  exponentiellement 
avec  l’accroissement  du  nornbre  d’echantillons  multivisees  utilises.  II  s’agit  d’un  moyen  tres 
efficace  d’ ameliorer  le  rendement  d’identification. 

Importance:  Les  resultats  de  la  methode  de  fusion  au  niveau  des  scores  indiquent  que  l’on  peut 
ameliorer  la  qualite  de  1’ identification  en  se  fondant  sur  la  quantite  des  donnees  plutot  que  sur 
leur  qualite,  ce  qui  pourrait  avoir  des  consequences  importantes  pour  les  applications  pratiques. 
D’une  part,  1’ apparition  de  technologies  de  capteurs  et  la  mise  en  service  de  plateformes 
multicapteurs  pour  la  surveillance  nous  permettent  d’obtenir  facilement  de  grandes  quantites  de 
donnees  de  qualite  mediocre  pour  le  renseignement;  alors  que  la  collecte  de  donnees  de  qualite 
constitue  encore  un  defi  technique  possiblement  plus  couteux.  La  capacite  de  maximiser 
l’exploitation  d’un  grand  nombre  de  donnees  disponibles  afin  d’ameliorer  le  rendement  de 
1’ identification  pourrait  avoir  un  effet  important  pour  1’ analyse  du  renseignement.  Cette  approche 
de  « la  quantite  plutot  que  la  qualite  »  pourrait  constituer  une  technologie  de  rupture  pour  les 
applications  scientifiques  et  techniques  en  defense  et  securite,  notamment  dans  le  domaine  du 
renseignement,  de  la  surveillance  et  de  la  reconnaissance.  On  peut  aussi  1’ employer  a  d’autres 
problemes  generaux  de  reconnaissance  de  forme,  par  exemple  la  reconnaissance  des  visages. 
Nous  avons  done  decouvert  une  propriete  particuliere  de  la  methode  de  fusion  au  niveau  des 
scores  :  elle  impartit  une  valeur  pratique  aux  donnees  de  signature  de  mauvaise  qualite  et  permet 
de  les  exploiter.  Notre  recherche  constitue  un  pas  important  pour  la  mise  au  point  d’algorithmes 
d’identification  des  objectifs. 


Perspectives:  Jusqu’a  maintenant,  nos  recherches  n’ont  porte  que  sur  1’ application  de  la  methode 
de  fusion  des  scores  a  1’ analyse  de  1’ identification  des  signatures  d’ objectifs  par  radar.  Pour  ce 
rapport,  nous  avons  analyse  des  signatures  d’objectifs  au  sol  dans  la  bande  X  (10  GHz)  et  nous 
avons  obtenu  des  resultats  exceptionnellement  bons.  Lors  de  travaux  anterieurs,  nous  avions 
etudie  les  signatures  d’aeronefs  en  vol  dans  la  bande  Ku  (16  GHz)  et  avions  egalement  obtenu 
d’excellents  resultats  d’identifications.  Les  recherches  sont  actuellement  etendues  au  probleme  de 
la  reconnaissance  des  visages.  Un  projet  de  recherche,  Demonstration  d’un  algorithme  et  d’un 
systeme  de  reconnaissance  des  visages  sur  video,  est  en  cours  qui  evaluera  la  methode  de  fusion 
au  niveau  des  scores,  des  informations  contenues  dans  des  images  video. 
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On  a  publie  tres  peu  de  travaux  theoriques  sur  la  methode  de  fusion  au  niveau  des  scores.  Lui 
donner  une  description  mathematique  plus  precise  et,  ainsi,  un  fondement  theorique  plus  solide 
constituerait  un  defi  et  une  occasion.  Realiser  davantage  de  recherche  fondamentale  serait 
desirable  et  aurait  une  valeur  strategique.  Une  meilleure  comprehension  conceptuelle  de  ses 
fondements  aurait  un  effet  important  pour  son  application  a  de  nouvelles  utilisations  et  a  d’  autres 
problemes  de  reconnaissance  de  forme,  particulierement  dans  les  domaines  de  la  science  et  de  la 
technologie  en  defense  et  en  securite. 
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1  Introduction 


Over  the  past  twenty  years,  an  extensive  research  effort  has  been  undertaken  in  developing 
algorithms  for  the  pattern  recognition  problem.  This  endeavour  is  particularly  relevant  and 
important  to  various  military  and  security  applications.  These  include  Combat  Identification, 
Aided  Target  Recognition,  Low  Probability  of  Intercept  radar  identification  and  human  biometric 
recognition.  Up  until  recently,  much  of  the  effort  has  been  devoted  to  achieve  high  identification 
accuracy  using  a  single  target  signature  as  input  in  the  identification  process.  The  motivation 
behind  the  single-look  approach  is  driven  partially  by  idealistic  factors.  The  objective  of  the 
identification  problem  was  initially  viewed  as  one  that  was  dealing  with  the  creation  of  a  model, 
which  given  a  minimum  amount  of  input  information,  would  be  able  to  generate  the  most 
accurate  identification  decision  outcome.  A  second  factor  was  practical  consideration.  When  the 
target  identification  development  effort  was  initiated  more  than  twenty  years  ago,  a  large  number 
of  radar  target  signature  data  was  not  readily  available.  Thus  it  would  be  sensible  to  develop  an 
algorithm  that  requires  only  a  minimum  amount  of  input  information,  but  achieving  the  maximum 
results.  But  it  was  found  that  single-input  identification  accuracy  was  very  much  dependent  on  the 
quality  of  the  input  target  signatures  used.  The  signature  quality  can  be  affected  by  parameters 
such  as  spatial  resolution,  target  motion  effect  and  various  environmental  factors.  With 
sufficiently  good  signature  quality,  correct  identification  rate  in  the  high  eighties  and  low  nineties 
percentage  can  be  achieved  readily.  However  in  some  applications,  for  example  Combat 
Identification,  identification  accuracy  better  than  99%  is  sought. 

To  improve  on  the  accuracy,  multi-look  fusion  methods  have  been  investigated,  in  which  a 
number  of  signatures  are  used  collectively  to  determine  the  identification  outcome.  Essentially, 
the  fusion  methods  make  attempts  to  reduce  the  level  of  uncertainty  in  the  decision  outcome  by 
maximizing  the  suitable  measures  of  evidence  [1],  There  have  been  a  number  of  different  multi¬ 
look  fusion  approaches  reported  in  the  radar  target  identification  problem,  and  in  human 
biometric  recognition,  for  example,  fusion  at  the  sample -input  level,  feature  level,  score  level  and 
decision  level  [2-12J.  Techniques  such  as  the  Majority-Vote  method,  the  Expectation- 
Maximization  method,  and  the  Dempster-Schafer  Rule  (Theory  of  Evidence)  are  some  of  the 
more  popular  multi-look  tools  used.  However,  virtually  all  multi-look  fusion  identification  studies 
reported  in  the  literature  focused  on  the  use  of  relatively  good  quality  target  signatures.  The 
objectives  of  these  studies  are  essentially  the  same,  trying  to  improve  the  correct  identification 
rate  from  the  high  80s  and  low  90s  to  over  99%. 

With  the  advent  of  surveillance  technology,  a  large  abundance  of  target  image  data  are  routinely 
available.  The  ability  to  maximize  the  exploitation  of  a  large  amount  of  available  data  has  become 
a  major  motivation  in  pursuing  multi-look  identification  methods.  However,  the  quality  of  the 
data  generated  in  large  volume  may  not  necessarily  be  adequate  to  provide  the  desired 
identification  performance  using  current  techniques  reported  in  the  literature. 

In  this  report,  a  different  perspective  on  the  multi-look  identification  problem  is  explored.  A 
multi-look  technique  known  as  the  score-level  fusion  method  is  investigated  in  this  work.  It  will 
be  shown  that  the  score-level  fusion  method  does  not  require  good  quality  signature  data; 
multiple  samples  of  marginal  quality  may  be  used  instead,  but  can  still  provide  highly  accurate 
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and  robust  identification.  A  qualitative  explanation  is  given  on  how  high  performance  is  achieved 
by  this  method.  Limitations  of  the  score-level  fusion  method  are  also  discussed. 

To  demonstrate  the  effectiveness  of  the  score-level  fusion  method,  a  detailed  comparative  study  is 
conducted  to  characterize  two  different  multi-look  fusion  methods.  The  majority-vote  method  and 
the  “score-level”  fusion  method  are  investigated;  their  identification  performance  are  assessed 
and  compared.  The  objective  of  the  comparative  study  is  to  highlight  the  practical  aspects  and  the 
technical  advantages  of  the  score-level  fusion  method  by  contrasting  its  performance  with  respect 
to  that  of  the  theoretically  well-established  majority-vote  method. 

The  majority-vote  method  is  a  well-known  fusion  method  commonly  used  in  many  different 
applications  in  pattern  recognition.  It  has  a  rigorous  theoretical  foundation  in  statistical  analysis 
[13]  [14]  [15].  On  the  other  hand,  the  “score-level”  fusion  method  has  been  deployed  in  multi¬ 
modal  biometric  identification  where  different  biometrics  are  fused  together  to  extract  an 
identification  decision  [12].  However,  analysis  of  the  fundamentals  of  this  fusion  method  is  not 
well  documented.  There  is  apparently  very  little  basic  research  being  carried  out  on  this  method 
so  far;  only  a  phenomenological  analysis  of  the  score-level  fusion  method  is  given  in  [16]  [17]. 
Empirical  characterization  of  the  score -level  method  will  also  be  given  here  to  provide  better 
insights  and  clearer  understanding  of  this  process.  There  are  a  number  of  variations  of  the  score- 
level  fusion  method;  the  “simple-average”  approach  is  investigated  in  this  study.  It  has  been 
found  that  the  simple-average  approach  gives  the  best  identification  results  in  biometric  studies 
[17]  [18], 

Experimental  High  Range  Resolution  (HRR)  profile  signatures  extracted  from  the  measured 
MSTAR  (Moving  and  Stationary  Target  Acquisition  and  Recognition)  ground  target  datasets  will 
be  used  to  evaluate  these  two  fusion  identification  methods.  HRR  signatures  are  widely  used  in 
target  recognition  for  moving  targets  [13]  [15].  The  quality  of  the  HRR  signatures  can  be  made  to 
vary;  this  facilitates  the  study  of  the  effect  of  data  quality  has  on  the  identification  results.  In 
particular,  emphasis  will  be  on  demonstrating  the  superior  identification  performance  of  the 
score -level  fusion  method  using  low  quality  target  signature  data. 
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2  Multi-Look  Identification  Methods 


2.1  Bayesian  Classifier 

A  target  classifier  identifies  an  unknown  target  by  computing  a  probability  value  to  quantify  how 
likely  the  input  target  will  match  with  one  of  many  targets  for  which  the  classifier  has  information 
stored  in  its  database.  Thus  the  design  of  a  classifier  involves  modeling  the  likelihood  of  the  input 
sample  to  match  information  stored  in  the  classifier,  determining  the  criteria  that  will  adequately 
distinguish  the  input  to  a  given  known  target  class  and  designing  the  identification  algorithm  to 
predict  an  outcome. 

The  Bayesian  classifier  will  be  employed  in  the  investigation  conducted  here.  It  is  a  widely  used 
classifier  in  pattern  recognition  applications  with  well-established  principles.  The  problem  of 
identification  is  basically  one  of  partitioning  targets  into  different  specific  types,  for  example: 
tanks,  armoured  vehicles  and  trucks  etc.  Ideally,  this  partitioning  is  to  be  arranged  so  that  none  of 
the  deduced  outcomes  is  ever  wrong.  More  realistically,  one  would  like  to  minimize  the  errors  in 
the  outcomes.  Thus,  the  problem  of  identification  is  reduced  to  a  problem  in  statistical  theory. 

The  computation  of  the  statistical  likelihood  of  an  event  is  given  by  the  Bayes  rule  [13J, 


P^cOj  I  x)  = 


p(x\col)P{coi) 

p(x ) 


(i) 


where  o>,  is  the  state  of  the  i-th  target  (e.g.,  tank,  armored  vehicle,  etc.),  x  is  the  prior  given 
information  of  the  targets  (e.g.,  observed  data  such  as  HRR  profiles),  P((p\x)  is  the  a  posteriori 
probability  (i.e.,  the  probability  that  the  state  0),  is  true  given  the  prior  information  x ),  p(x\0J,)  is  the 
state  conditional  probability  density  (i.e.,  the  probability  distribution  function),  P(O)j)  is  the  a 
priori  probability  (i.e.,  it  represents  the  state  of  our  prior  knowledge  about  the  target  before  we 
have  analyzed  the  data)  and  p(x)  =  Y.l(p(x\o),)P(oJl))  in  the  denominator  is  a  normalization  factor. 

In  the  identification  problem,  the  decision  to  declare  an  unknown  target  x  to  be  the  known  state 
co,„  is  defined  by 

P(C0m  I  x)  =  max,.  P(ftj  I  x)  (2) 

That  is,  the  state  of  the  target  co,„  (i.e.,  a  specific  target  type)  that  achieves  the  largest  probability 
value  is  declared  as  the  identification  outcome  that  matches  the  unknown  input  x.  This  is  known 
as  the  nearest-neigbour  rule  [13]. 


2.2  Majority-Vote  Method 

An  obvious  extension  to  the  Bayesian  decision  making  process  is  to  identify  the  unknown  target 
by  assigning  it  to  the  target  label  that  occurs  most  frequently  among  k  submitted  samples  of  the 
unknown  target.  This  is  the  essence  of  the  majority-vote  method.  The  majority-vote  method  in 
fact  has  a  rigorous  mathematical  foundation;  it  can  be  expressed  as  a  binominal  distribution 
derived  from  random  variable  analysis  in  probability  theory  [15];  i.e., 
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PM,)=  Z  r\p(a>Jxr(l-P(a,Jx)f-n  (3) 

n=(k+l)!2\nJ 

where  P(cq„\x)  is  the  a  posteriori  probability  given  by  Equation  (2)  for  a  single-input  sample  (i.e., 
single-look)  and  the  indices  k  and  n  are  the  numbers  of  multi-look  input  samples.  The  bracketed 
quantity  is  the  binominal  coefficient.  The  mathematical  expression  given  by  Equation  (3)  is  also 
known  as  the  k-nearest-neighbour  rule  [13].  The  majority-vote  method  is  a  widely  used  multi¬ 
look  fusion  method  in  pattern  recognition  applications  [14].  It  is  a  physically  intuitive  process  that 
can  be  understood  easily,  and  it  has  a  very  well-established  theoretical  background. 

2.3  Score-Level  Fusion  Method 

The  score-level  fusion  method  has  already  been  used  in  multi-modal  biometric  identification  [12J. 
But  there  is  very  little  detailed  analytical  work  in  the  literature  on  this  method.  A  number  of 
analyses  of  the  score-level  fusion  method  are  given  in  [16]  [17]  [19].  However,  they  are 
phenomenological  characterization  in  nature.  The  score-level  fusion  method  discussed  in  this 
work  takes  the  form  of  a  simple  Bayes  average  of  the  a  posteriori  probabilities  from  k  input 
samples  computed  by  Equation  (1);  no  decision  is  computed  from  each  input  sample.  The  a 
posteriori  probability  for  score-level  fusion  is  given  by, 

1  k 

P.SI.F  (fy  I  -*i  '  "  Xk  )  —  T  I  Xn  )  (4) 

k  n= 1 

The  identification  decision  is  then  computed  by, 

Pslf  (®m  I  A  '  "  Xk)  ~  PSLF  (  I  Aj  •  •  •  xk )  (5) 

It  should  be  noted  that  the  simple  Bayes  average  is  only  one  of  many  ways  of  conducting  score- 
level  fusion  [12];  but  it  seems  to  provide  the  best  performance,  at  least  in  biometric  identification 
applications  [17]  [18].  A  more  detailed  discussion  from  analysis  using  measured  data  will  be 
given  in  Section  5.  In  brief,  the  score-level  fusion  method  can  be  summarized  as  a  form  of 
implementing  the  Bayes  decision  rule  by  minimizing  the  classification  errors  over  and  above  the 
Bayes  rate  due  to  errors  in  the  estimate  of  the  a  posteriori  probability.  The  larger  the  variance  of 
the  error  distribution,  the  larger  the  classification  error  becomes.  The  score -level  fusion  method 
reduces  the  variance  and  hence  diminishes  the  classification  error  [16]. 
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3  MSTAR  (Moving  and  Stationary  Targets  Acquisition 
and  Recognition)  Datasets 


The  performance  of  the  majority-vote  method  and  the  score -level  fusion  method  will  be  assessed 
using  measured  target  signature  data  from  the  MSTAR  datasets.  These  datasets  contain  measured 
SAR  (Synthetic  Aperture  Radar)  images  of  10  ground  vehicles  and  one  man-made  canonical 
target  (SLICY),  for  a  total  of  1 1  targets.  The  details  of  the  MSTAR  datasets  are  tabulated  in  Table 
1 ;  photographs  of  the  target  types  are  shown  in  Figure  1 .  The  SAR  images  were  collected  using 
the  spotlight-SAR  mode  with  an  angular  coverage  from  1  to  360  degrees  in  azimuth.  The  SAR- 
image  data  collected  at  depression  angles  15  and  17  degrees  are  used  for  identification  analysis, 
one  as  test  set  and  the  other  as  library  set  in  the  classifier.  Two  of  the  targets,  BPM2  (armoured 
personnel  carrier)  and  T72  (battle  tank),  have  3  variants  each;  that  is,  there  are  three  versions  of 
the  same  target  with  slightly  different  target  configurations  [21].  All  images  have  equal  down- 
range  resolution  Ard  and  cross-range  resolution  zlrr;  that  is,  Ard  =  Arc  =  0.305  m.  Each  SAR  image 
represents  an  angular  aperture  of, 

A6>  =  -  ^  =  2.9  deg  ,  (6) 

2fArc 

where /=  9.6  GFIz  is  the  centre  frequency  of  the  radar  waveform  and  c  is  the  speed  of  light.  The 
spotSAR  radar  was  flown  several  times  around  the  targets,  providing  multiple  360-degree 
azimuth  coverage  in  the  MSTAR  measurements. 


Table  1  Target  types  in  the  MSTAR  data  set  used  in  the  identification  analysis 


Target  types 

No.  SAR  images 
Test  set 

(15  deg.  elevation) 

No.  of  SAR  images 

Library  set 

(17  deg.  elevation) 

Pixel  size  of 
SAR  images 

No.  of  valid  HRR 
signatures  yielded 
per  SAR  image  [14] 

BMP2(9563) 

195 

128x128 

96 

BMP2(9566) 

196 

128x128 

96 

BMP2(c21) 

196 

233 

128x128 

96 

T72(132) 

196 

232 

128x128 

96 

T72(8 12) 

195 

128x128 

96 

T72(s7) 

191 

128x128 

96 

BTR70 

196 

233 

128x128 

96 

BTR60 

195 

256 

128x128 

96 

BRDM2 

274 

298 

128x129 

96 

ZSU 

274 

299 

158x158 

120 

T62 

273 

299 

172x173 

136 

7.11. 

274 

299 

192x193 

152 

2S1 

274 

299 

158x158 

120 

D7 

274 

299 

177x177 

136 

SLICY 

274 

54x54 

48 
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2S1  D7  SLICY 

Figure  1  Photographs  of  the  ground-vehicle  targets  from  the  MSTAR  datasets. 

3.1  Extracting  High  Range  Resolution  profiles  as  signatures 

High  Range  Resolution  (HRR)  profiles  as  target  signatures  can  be  extracted  from  the  MSTAR 
SAR  images  [20].  The  amplitudes  of  the  extracted  HRR  profiles  from  the  SAR  images  can  vary 
considerably  with  changing  azimuth  angle.  HRR-profile  signatures  of  different  qualities  can 
hence  be  generated  by  averaging  a  number  of  HRR  profiles  over  different  subtended  azimuth 
aperture  sizes  [20] .  The  fluctuation  in  the  amplitude  as  a  function  of  the  azimuth  angle  is  a  well- 
known  phenomenon  [22];  it  is  known  as  the  glinting  effect  in  which  the  phases  of  two  or  more 
scatterers  residing  in  the  same  range  bin  interfere  with  one  another  as  the  azimuth  angle  is  rotated 
slightly.  It  has  been  reported  that  the  fluctuation  can  be  mitigated  effectively  by  averaging  the 
HRR  profiles  over  a  large  azimuth  angle  [  1 1]  [23] .  Hence  four  sets  of  HRR  signatures  of  different 
qualities  are  created  by  extracting  8,  4,  2,  and  1  averaged  HRR  profiles  from  each  SAR  image. 
These  correspond  to  signatures  subtending  an  azimuth  aperture  of  0.36,  0.73,  1.45  and  2.9 
degrees  respectively. 

Figures  2  and  3  show  sequences  of  4  sets  of  HRR  signatures  of  different  qualities  for  the  BMP2 
armoured  personnel  carrier  and  the  T72  battle  tank  respectively.  It  can  be  seen  that  the  HRR 
profile-to-profile  variability  is  very  large  when  the  azimuth  angle  subtended  is  small;  i.e.,  when  8 
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HRR  profiles  are  extracted  from  1  SAR  image.  This  is  illustrated  in  Figures  2a  and  3a.  As  the 
subtended  azimuth  angle  is  increased  in  averaging  the  HRR  profiles,  the  profile-to-profile 
variability  subsides  progressively.  This  is  illustrated  in  Figures  2c  and  3c  (i.e.,  2  HRR  profiles 
created  from  1  SAR  image).  When  a  HRR  profile  is  generated  by  averaging  over  the  whole  SAR 
image  (2.9  degrees  in  azimuth),  the  amplitude  fluctuations  in  the  averaged  HRR  profiles  have 
subsided  very  considerably  such  that  the  HRR  profiles  between  adjacent  SAR  images  look  very 
similar.  This  is  illustrated  in  Figures  2d  and  3d.  Thus,  the  quality  of  the  signatures  can  be 
characterized  qualitatively  by  the  amount  of  variation  from  profile  to  profile  in  the  HRR 
signatures,  as  illustrated  in  Figures  2  and  3.  In  Section  4.4,  a  more  quantitative  characterization  of 
the  quality  of  target  signatures  will  be  given;  it  will  be  shown  that  the  correct  identification  rate  is 
dependent  on  the  amount  of  profile-to-profile  variability  in  the  HRR  signatures. 
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T72-132 


Figure  2  HRR  profile  variability  as  a  function  of  azimuth  angle  subtended  for  the  T72  battle 
tank.  Each  HRR  profile  is  averaged  over:  a)  0.36  degrees,  b)  0.73  degrees,  c)  1.45 
degrees,  d)  2.9  degrees  in  azimuth  angle.  In  d),  averaged  HRR  profiles  from  2  adjacent 
SAR  images  are  shown. 
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BTR-70 


BTR-70 


BTR-70 


BTR-70  1st  BTR-70  2nd 


Figure  3  HRR  profile  variability  as  a  function  of  azimuth  angle  subtended  for  the  BTR70 
armoured  carrier.  Each  HRR  profile  is  averaged  over:  a)  0.36  degrees,  b)  0.73 
degrees,  c)  1.45  degrees,  d)  2.9  degrees  in  azimuth  angle.  In  d),  averaged  HRR 
profiles  from  2  adjacent  SAR  images  are  shown. 
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4  Target  Identification 


The  identification  process  will  be  computed  using  the  Bayesian  classifier  described  in  Section 
2.1.  To  generate  an  algorithm  for  computing  the  decision  outcome,  Bayes  rule  given  by  Equation 
(1)  can  be  rewritten  as  [24], 

P(CO,  I  x)  oc  p(x  I  CO,).  (7) 


This  is  because  p(x)  in  the  denominator  of  Equation  (1)  is  simply  a  normalization  constant. 
Furthermore,  in  typical  pattern  recognition  problems,  we  assume  a  uniform  a  priori  probability 
Pico,).  This  means  the  unknown  target  is  assumed  equally  likely  to  be  any  one  of  the  possible 
targets  in  our  knowledge.  Therefore,  P{  co,)  in  Equation  (1)  is  identical  for  all  unknown  input 
targets  and  it  is  just  a  constant.  Thus  according  to  the  Bayes  rule,  the  a  posteriori  probability 
Pico, lx)  which  provides  the  identification  decision  about  the  unknown  target  is  reduced  to 
computing  the  probability  distribution  function,  pix I  CO,)  as  given  in  Equation  (7).  However,  in 
most  pattern  recognition  analysis,  there  is  not  enough  information  from  the  data  to  determine  a 
probability  distribution  function  with  well-defined  parameters.  In  most  cases,  it  has  been  found 
that  probability  distribution  functions  that  do  not  make  any  assumption  about  the  underlying 
statistical  distribution  can  be  estimated  [  1 3]  [25] .  The  probability  distribution  function  that  is  of 
great  importance  and  usefulness  is  the  Gaussian  distribution, 


g(x) 


I  v*  1 02  _\«/2 

I  £  I  (2x) 


“(x-|i)r£  ‘(x-[i) 


(8) 


where  x  is  a  signature  vector  (n  dimensions),  |X  is  the  mean  signature  vector  (n  dimensions),  L  is 
the  covariance  matrix  (n  x  n  dimensions),  El  is  the  determinant  and  the  superscript  T  denotes 
transpose  of  a  vector.  In  the  context  of  the  HRR-profile  analysis,  the  vector  x  is  the  input 
unknown  HRR  profile  with  n  range  bins,  |i  is  the  library  reference  HRR  profile  in  the  classifier. 
The  Gaussian  distribution  function  is  a  reasonable  assumption  and  it  satisfies  the  Central  Limit 
Theorem  [25]  which  states  that  the  sum  of  independent  distributed  random  variables,  under  very 
general  conditions,  has  a  distribution  converging  to  the  normal  distribution  function  as  given  by 
Equation  (8).  Furthermore,  it  is  assumed  that  the  features  in  a  HRR  profile  are  statistically 
independent;  that  is,  the  scattering  peaks  located  at  different  range  bins  are  independent  from  one 
another  in  the  formation  of  the  HRR  profile  [26] .  With  the  assumption  of  statistically  independent 
features  within  a  signature  vector  (e.g.,  a  HRR  profile),  all  the  off-diagonal  elements  of  the 
variance  E  are  zero  and  the  diagonal  elements  are  a  constant  [13].  As  a  result,  the  quadratic  factor 
in  Equation  (8),  ( x — (0. f 7( x — (X) ,  is  the  principal  factor  emerging  from  the  Gaussian  distribution 
function  that  forms  the  basis  of  a  target  classifier;  it  is  known  as  the  minimum  distance  classifier. 


4.1  Cross-correlation  Bayesian  classifier 

To  classify  an  unknown  input  vector  x,  the  Euclidean  distance  llx-|ill  from  x  to  each  of  the  mean 
vectors  |i  in  the  classifier’ s  library  is  computed.  The  mean  vector  |X  belonging  to  target  class  t  that 
produces  the  shortest  Euclidean  distance  with  the  unknown  vector  x  is  then  assigned  as  the 
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identified  target.  The  cross-correlation  procedure  is  one  of  many  ways  of  working  with  the 
minimum  distance  classifier.  This  can  be  seen  by  expanding  the  quadratic  factor  in  Equation  (8), 

(x-|l)r(x-|j,)  =  xrx  -  2|0,'rx  +  .  (9) 

The  cross-correlation  is  then  given  by, 

C  =  \lTx  =  ^  [L(n)  x(n) .  (10) 

n 


Note  that  if  the  cross-correlation  coefficient,  C  is  maximized  (that  is,  maximum  overlap  between 
the  vectors  x  and  |x),  then  (x-p)  '(x-p)  in  Equation  (9)  is  minimized.  A  normalized  form  of  the 
cross-correlation  procedure  is  given  by  the  Cauchy-Schwartz  Inequality  [13], 


C(x(m),(i.) 


2j(x(«  —  m)  |X(n)) 

n _ 


2>d02 

V  n 


\  1/2 

) 


Inw2 

V  n 


\  1/2 


<  1  . 


(11) 


The  variable  m  is  a  shifting  parameter  for  sliding  one  vector  over  the  other  to  maximize  the  cross¬ 
correlation  coefficient  C.  This  normalized  form  of  the  cross-correlation  method  is  used  in  the 
identification  analysis  here. 

In  the  cross-correlation  process,  the  unknown  input  HRR  profile,  x  is  correlated  with  respect  to 
the  library  templates  p,_,  to  find  the  maximum  cross-correlation  response  within  each  target  class 
t;  i.e., 


R(x,[ir)  =  max;(C(x,|if  ■) . 


(12) 


The  subscript  i  denotes  the  i-th  library  template  in  target  class  t.  There  are  N  templates  in  each 
target  class  covering  target  azimuth  from  0  to  360  degrees  (i  -  1,  . . .,  AO-  Correlation  of  the  input 
test  signature  against  all  N  templates  in  each  target  class  t  is  carried  out,  and  this  is  repeated  for 
all  target  classes  in  the  classifier.  A  set  of  R(x,  p,)  values  is  then  created,  one  value  for  each  target 
class  t.  Finally,  to  determine  the  identification  of  the  test  input  signature,  a  search  for  the 
maximum  cross-correlation  response  R(x,  p;)  among  all  target  classes  is  conducted.  The  target 
class  label  T  that  corresponds  to  the  maximum  correlation  response  is  assigned  as  the  identified 
target  class  for  the  test  input, 

T  =  arg(max^  R(x,  \x,t )) .  (13) 


4.2  Extended  Operating  Conditions 

In  order  to  make  decisions  that  are  meaningful  in  the  identification  analysis,  realistic  conditions 
must  be  incorporated  in  the  identification  process.  The  Extended  Operating  Conditions  (EOC)  are 
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a  set  of  rules  that  allow  the  testing  of  the  target  classifier  under  more  realistic  real-world 
scenarios.  They  are  defined  as  operating  conditions  that  differ  significantly  from  the  trained 
conditions  of  the  classifier  [21]  [27].  This  allows  more  rigorous  tests  to  be  conducted  to  assess  the 
identification  algorithm.  The  EOCs  as  outlined  in  references  21  and  27  are  applied  here  in 
examining  the  identification  algorithm's  performance.  The  extended  operating  conditions  are 
listed  as  follows: 

1.  Data  measured  at  17-degree  depression  angle  are  used  as  library  data  in  the  classifier,  and 
data  at  15-degree  depression  angle  are  used  as  unknown  inputs  for  testing  (that  is  to  say, 
identification  is  performed  using  data  that  are  “away  from  the  trained”  conditions). 

2.  No  knowledge  of  target  aspect  information  of  the  unknown  test  inputs  is  assumed. 
Identification  evaluation  is  performed  over  360  degrees  in  azimuth.  In  other  words,  the 
algorithm  must  be  able  to  handle  an  ambiguous  target  pose. 

3.  Only  one  variant  of  the  BMP2  (armoured  personnel  carrier)  and  one  variant  of  the  T72 
(battle  tank)  are  included  as  the  library  database  in  the  classifier.  Three  variants  of  the 
BMP2  and  three  variants  of  the  T72  are  used  as  unknown  test  targets  (see  Table  1). 

A  11 -class  forced-decision  classifier  is  used  to  provide  a  comparison  of  the  identification 
performance  between  the  majority-vote  method  and  the  score -level  fusion  method. 

4.3  Random  aspect  input  signature  sampling 

As  discussed  in  Section  3,  the  MSTAR  datasets  were  collected  at  all  azimuth  aspects;  i.e., 
covering  360  degrees.  An  example  of  the  azimuth  distribution  of  the  MSTAR  data  collected  for 
the  BTR70  (armoured  personnel  carrier)  is  shown  in  Figure  4a.  Random  sequences  of  the  test 
input  signatures  in  azimuth  are  selected  for  identification  analysis;  an  example  of  a  randomized 
azimuth  sequence  is  illustrated  in  Figure  4b. 

The  random  selection  of  azimuth  aspects  permits  more  realistic  and  flexible  conditions  to  be 
analyzed  in  the  identification  process.  For  example,  real-world  data  could  be  collected  from 
multi-platform  sensors  that  are  distributed  over  a  wide  area  spatially  or  from  an  accumulation  of 
data  by  a  single  sensor  collected  over  an  extended  period  of  time  during  a  long  surveillance 
mission  [9].  Thus,  any  available  data  spanning  over  a  large  azimuth  region  due  to  spatial  diversity 
or  temporal  diversity  can  be  utilized. 

Technically,  the  random  azimuth  selection  of  the  input  test  samples  is  also  consistent  with  the 
underlying  assumptions  used  in  the  identification  analysis.  For  example,  the  Gaussian  distribution 
used  in  the  Bayesian  classifier,  Equation  (8),  implicitly  implied  a  random  sampling  process,  and 
the  majority-vote  method  is  based  on  random  sampling  statistics.  Hence,  the  random-aspect 
signature  sequence  used  in  multi-look  sampling  is  a  logical  choice  and  provides  a  consistent, 
valid  basis  for  the  multi-look  identification  analysis. 
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time  (arb.  unit)  time  (arb.  unit) 


Figure  4  Examples  of  a)  a  multi-look  sequence  from  consecutive  aspect  angle  data  as  taken  in 
the  MSTAR  measurements ,  b)  a  multi-aspect  sequence  (dashed  box)  from  randomized 
aspect  angle  data. 

4.4  Quantifying  the  quality  of  the  HRR-profile  signatures 

In  Section  3.1,  the  quality  of  the  HRR-profile  signature  data  is  characterized  qualitatively  by  the 
profile-to-profile  variability.  Intuitively,  it  is  expected  that  better  the  data  quality,  more  accurate 
the  identification  would  be.  Thus,  the  correct  identification  rate  can  be  used  as  an  indicator  for 
assessing  the  signature  quality.  Using  the  Bayesian  classifier  and  procedures  described  in 
Sections  4.1,  4.2  and  4.3,  the  quality  of  the  HRR  signature  datasets  can  be  characterized  more 
quantitatively  from  the  identification  results. 

Single  HRR  signature  is  used  as  unknown  input  into  the  Bayesian  classifier;  an  identification  is 
made  for  each  test  input  signature.  In  the  identification  process,  the  qualities  of  the  signatures  in 
the  test  set  and  in  the  classifier  library  are  the  same;  that  is  to  say,  the  test  set  and  the  train  set  are 
consistent  and  compatible  with  each  other.  Figure  5  shows  the  correct  identification  rate  as  a 
function  of  the  signature  quality  expressed  in  terms  of  the  subtended  azimuth  angle  of  the  HRR 
profiles.  It  can  be  seen  that  as  the  quality  of  the  signature  improves,  the  correct  identification  rate 
increases  as  a  result.  The  correct  identification  rate  is  defined  as  the  ratio  the  number  of  correctly 
identified  input  signatures  to  the  total  number  of  signatures  tested.  Table  2  shows  the 
identification  results  in  the  form  of  a  confusion  matrix  for  the  signature  set  that  has  the  poorest 
quality,  e.g.,  the  HRR  signatures  with  the  smallest  subtended  azimuth  angle  (0.36  degrees).  The 
correct  identification  rate  Pid  in  this  case  is  only  37%,  or  9581  correct  identifications  out  of  25624 
input  signatures  tested.  Figure  5  provides  a  summarized  view  of  the  effect  of  the  quality  of  the 
data  on  the  identification  process.  It  is  also  seen  from  Figure  5  that  even  for  the  best  quality  data 
case,  the  quality  is  still  not  adequate  enough  to  achieve  high  accuracy;  the  correct  identification 
rate  achieved  is  only  72%.  This  underscores  the  short-coming  of  the  single-look  approach  and 
motivates  the  investigation  into  multi-look  identification  procedures. 
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Figure  5  Correct  identification  rate  Pm  as  a  function  of  data  quality. 


Table  2  Single-look  identification  results  from  a  11 -class  classifier  for  the  dataset  with  the 
poorest  target  signature  quality 


Library 

MP2 

T72  BTR70 

SLICY 

BTR60 

BRDM2 

ZSU 

T62 

ZIL 

2S1 

D7 

(c21) 

(132) 

Test  targets 

BMP 2 (9563) 

554 

143 

188 

21 

184 

195 

26 

33 

81 

117 

18 

BMP 2 (9566) 

503 

139 

171 

22 

232 

237 

24 

21 

69 

127 

23 

BMP 2 ( c2 1 ) 

550 

171 

167 

18 

167 

177 

21 

53 

83 

125 

36 

T72 (132) 

132 

554 

63 

7 

97 

57 

107 

95 

189 

146 

121 

T72 (812) 

107 

381 

54 

5 

84 

83 

107 

164 

193 

178 

204 

T72 ( s7 ) 

143 

378 

75 

17 

180 

98 

136 

93 

163 

162 

83 

BTR7  0 

190 

53 

640 

0 

193 

246 

13 

30 

49 

136 

18 

BTR60 

131 

172 

158 

2 

448 

152 

57 

88 

142 

153 

57 

BRDM2 

260 

95 

331 

20 

241 

949 

25 

26 

86 

133 

26 

ZSU 

13 

90 

5 

10 

27 

9 

1275 

196 

183 

117 

267 

T62 

64 

168 

30 

23 

142 

59 

388 

733 

231 

107 

239 

ZIL 

91 

208 

67 

20 

130 

87 

197 

162 

818 

204 

208 

2S1 

175 

208 

114 

12 

139 

96 

137 

188 

182 

798 

143 

D7 

21 

75 

10 

28 

58 

13 

335 

323 

176 

153 

1000 

total  no  of 

test 

inputs 

=  25624 

no  of  correct  ID 

=  9581 

correct  ID 

rate 

=  0.3739 
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5  Identification  Results:  Majority-Vote  vs.  Score-Level 
Fusion 


Over  the  past  10  years  or  so,  the  biometrics  community  has  been  working  on  the  identification 
problem  using  various  multi-look  fusion  schemes  to  combine  different  biometric  information  in 
the  identification  process,  e.g.,  face  images,  fingerprints,  hand  geometry,  voice  patterns  and  iris 
scans.  There  are  generally  four  different  multi-look  fusion  approaches:  sample-level,  feature- 
level,  score-level  and  decision-level  fusions  [9] .  Sample-level  fusion  combines  a  number  of  target 
signatures  or  images  together;  this  is  illustrated  in  Figure  6a.  To  a  certain  extent,  a  sample-level 
fusion  has  already  been  applied  in  the  non-coherent  averaging  of  the  FIRR  profiles  in  Section  3.1 
to  generate  the  different  grades  of  signature  quality  by  varying  the  profile -to-profile  variability.  It 
can  be  seen  from  Figure  5  that  improvement  in  the  correct  identification  rate  can  indeed  be 
achieved  by  fusing  a  number  of  samples  together.  Feature-level  fusion  involves  extraction  of 
relevant  information  from  the  target  signatures.  The  set  of  extracted  features  are  then  used  as 
input  parameters  in  the  identification  process;  this  is  illustrated  in  Figure  6b.  In  target 
identification  analysis  using  FIRR  profiles  as  target  signatures,  the  normal  practice  is  to  use  the 
whole  HRR  profile  as  a  target  feature;  hence  feature-level  fusion  is  not  used. 

In  this  section,  a  comparative  study  is  conducted  to  evaluate  the  performance  of  the  two 
remaining  multi-look  identification  methods,  majority-vote  fusion  and  score-level  fusion.  The 
results  from  each  of  these  two  methods  will  be  discussed  and  compared.  The  advantages  of  using 
the  score-level  fusion  method  will  then  be  assessed  quantitatively.  It  will  be  shown  that  the  score- 
level  fusion  method  is  the  more  accurate  and  versatile  of  the  two,  and  this  method  has  a 
significant  practical  implication  in  applications. 


Figure  6  Schematics  of  the  sample-level  (a)  and  feature-level  (b)  fusion  methods 


5.1  Majority-Vote  Method 

The  majority-vote  method  is  a  data  fusion  procedure  that  makes  a  decision  through  consensus 
from  multiple  outcomes  at  the  decision  level  of  the  classifiers;  this  is  illustrated  schematically  in 
Figure  7a.  An  identification  outcome  from  each  single  test  sample  in  a  multi-look  sequence  is 
first  computed  using  Equation  (13).  Given  a  sequence  of  k  input  unknown  signatures,  there  are  k 
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target  labels  as  identification  outputs.  The  majority-vote  method  simply  finds  the  target  label  that 
appears  the  largest  number  of  times  in  the  ^-multi-look  sequence.  It  should  be  noted  that  an  odd 
number  of  multi-look  samples  is  usually  used  in  majority- vote;  this  is  to  ensure  that  a  tie  will  be 
avoided  in  the  vote  counts.  A  minimum  majority  is  required  to  declare  a  decision;  the  minimum 
majority  n  is  given  by  n  =  (k+ 1  )/2.  As  an  example,  if  k  =  7,  then  the  minimum  number  of  votes 
needed  to  declare  an  identification  is  n  -  4.  If  n  <  4,  then  a  “no-majority”  response  is  given  as 
output.  Note  that  a  no-majority  response  is  counted  as  incorrect  identification  in  the  majority-vote 
evaluation  carried  out  in  this  study. 


Figure  7  Schematics  of  the  majority-vote  (a)  and  score-level  (b)  fusion  methods. 


Figure  8  shows  the  majority-vote  identification  results  from  measured  data  (dashed  curves)  as  a 
function  of  the  number  of  signatures  used  in  a  multi-look  sequence  from  an  11 -class  target 
classifier.  The  top  dashed  curve  corresponds  to  results  from  the  dataset  with  the  best  signature 
quality;  the  bottom  dashed  curve  represents  results  from  the  dataset  that  has  the  poorest  signature 
quality.  There  are  a  number  of  notable  results  that  can  be  deduced  from  Figure  8. 

For  data  quality  that  is  good  enough  to  achieve  a  correct  identification  rate  Pld  of  greater  than 
50%  in  single-look  input  ( k  =1),  the  majority-vote  identification  results  improve  quite 
considerably  with  an  increasing  number  of  signature  samples  in  the  multi-look  sequence.  These 
are  illustrated  by  the  upper  two  dashed  curves  in  Figure  8.  As  the  data  quality  deteriorates  to  such 
levels  that  the  correct  identification  rate  is  below  50%,  an  increase  in  the  number  of  multi-look 
samples  does  not  improve  the  correct  identification  rate.  In  fact,  the  identification  results  are 
worse  using  the  majority-vote  method.  These  are  shown  in  the  two  lower  dashed  curves. 
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Figure  8  Majority-vote  identification  results.  Correct  identification  rate,  P„i  as  a  function  of  the 
number  of  samples,  N  in  the  multi-look  sequence  for  4  different  signature  data 
qualities.  Solid  curves  =  theoretical  predictions;  dashed  curves  =  measured  data. 


The  majority-vote  method  has  a  rigorous  theoretical  background;  it  can  be  described  by  a 
mathematical  formula  as  given  by  Equation  (3).  These  predictions  are  given  by  the  solid  curves  in 
Figure  8.  The  majority-vote  results  obtained  from  measured  data  (dashed  curves)  are  compared 
with  the  theoretical  predictions  (solid  curves).  It  can  be  seen  from  Figure  8  that  the  difference  in 
the  correct  identification  rate  is  less  than  6%  between  the  measured  results  and  the  theoretical 
predictions  in  all  4  cases.  This  indicates  that  the  results  from  the  measured  data  are  reasonable  in 
providing  a  validation  of  the  notion  of  different  signature  qualities  among  the  4  datasets. 
Furthermore,  these  results  reveal  a  serious  limitation  of  the  majority-vote  method;  its  performance 
is  seen  as  being  constrained  by  the  quality  of  the  target  signatures.  The  results  in  Figure  8  suggest 
that  in  order  for  the  majority-vote  method  to  be  useful,  the  quality  of  the  target  signatures  must  be 
good  enough  to  achieve  a  single-look  correct  identification  rate  of  at  least  70%  or  better. 
Otherwise,  correct  identification  rate  of  greater  than  90%  may  not  be  realizable. 

5.2  Score-Level  Fusion  Method 

The  score-level  fusion  method  combines  the  cross-correlation  scores  from  multiple  Bayesian 
classifiers  as  described  in  Section  4.1.  That  is  to  say,  the  cross-correlation  score  from  each  input 
sample  is  computed,  but  no  identification  decision  is  made.  The  cross-correlation  scores  from  k 
input  samples  are  instead  combined  using  Equation  (4)  to  generate  a  single  effective  score  for 
each  target  class  that  are  stored  in  the  classifier.  Then,  an  identification  decision  is  made  by 
finding  the  target  label  that  has  the  largest  score  as  given  by  Equation  (5).  A  schematic  of  the 
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score -level  fusion  process  is  shown  in  Figure  7b.  Using  Equation  (4),  a  simple-average  cross¬ 
correlation  score  from  a  series  of  k  input  F1RR  signatures  is  computed  for  each  target  class  t  in  the 
classifier;  i.e., 

W  =  }  £«(*„. H,)>  (14) 

R  n= 1 

where  R(xn,  p,)  is  the  maximum  cross-correlation  response  for  each  of  the  test  inputs  x„,  n  -  1,  .. 
k  as  given  by  Equation  ( 12).  This  procedure  is  repeated  for  all  target  classes  in  the  classifier,  e.g., 
t  -  1,  . 1 1  in  the  1 1 -target  classifier.  The  identification  outcome  is  then  determined  by  the  target 
class  label  T  that  corresponds  to  the  maximum  multi-look  response  value;  i.e., , 

T  =  arg(max,(7?,».  (15) 


5.3  Score-Level  Fusion  identification  results 

Using  the  above  procedure,  identification  results  using  the  score-level  fusion  method  are  shown 
in  Figure  9  given  by  the  solid  curves.  It  can  be  seen  that  the  identification  results  for  the  4 
different  signature  qualities  are  exceedingly  good  when  a  sufficiently  large  number  of  multi-look 
samples  is  used.  For  comparison,  the  results  from  the  majority- vote  method  are  also  plotted 
(dashed  curves)  in  Figure  9.  It  is  seen  that  the  score-level  fusion  method  is  significantly  better 
than  the  majority-vote  method  in  identification  performance  for  all  four  signature  qualities.  It  is  of 
particular  interest  to  note  that  the  correct  identification  rate  using  poor  quality  signature  can  be 
improved  very  dramatically,  from  37%  in  single-look  to  over  97%  using  a  large  number  of  look 
samples  (>20).  This  is  highlighted  by  the  heavy  solid  curve  in  Figure  9.  The  good  identification 
performance  of  the  score-level  fusion  method  has  some  interesting  implications.  Firstly,  the 
results  suggest  that  quantity  can  be  used  to  replace  quality  of  the  data  in  improving  the 
identification  accuracy,  e.g.,  using  a  large  number  of  data  samples  of  marginal  quality  in  a  multi¬ 
look  sequence.  Secondly,  signature  data  of  marginal  quality  can  still  be  used  meaningfully  in 
target  identification;  this  means  data  quality  may  not  be  as  important  a  requirement  as  one  would 
expect  intuitively. 
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Figure  9  Score-level  fusion  identification  results  from  measured  data  (solid  curves).  Correct 
identification  rate,  P„i  as  a  function  of  the  number  of  samples,  N  in  the  multi-look 
sequence.  The  dashed  curves  are  the  majority-vote  results  from  measured  data  for 
comparison 
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6  A  Qualitative  Analysis  of  the  Score-Level  Fusion 
Method 


It  has  already  been  seen  in  the  majority-vote  method  that  multi-look  sampling  can  produce  more 
accurate  identification  than  single-look  sampling.  In  the  majority-vote  case,  the  increase  in 
accuracy  is  a  consequence  of  the  fundamental  property  of  the  binomial  distribution  function  given 
in  Equation  (3).  In  more  qualitative  terms,  the  majority-vote  method  is  expected  to  perform  better 
than  the  best  single-look  classifier  in  identifying  an  unknown  input  target,  provided  that  the  target 
signature  quality  is  sufficiently  adequate.  In  brief,  the  essence  of  the  majority-vote  method  is  to 
reduce  the  level  of  uncertainty  through  maximizing  suitable  measures  of  evidence  by  monitoring 
how  frequently  a  particular  outcome  occurs.  But  in  the  score -level  fusion  method,  it  is  not  clear 
how  and  why  a  multi-look  sequence  of  inputs  produces  significantly  better  correct  identification, 
even  when  the  quality  of  the  signatures  is  poor.  In  the  limited  literature  available  that  describes 
the  score-level  fusion  method,  the  underlying  assumption  is  that  the  a  posterior  probability  is 
affected  by  its  associated  variance  [16]  [17].  The  larger  this  variance,  the  larger  the  identification 
error  is  produced  as  a  result.  The  score-level  fusion  method  reduces  this  variance  and 
consequently,  diminishes  the  identification  error.  In  other  words,  the  target  label  error  from  the 
score -level  fusion  will  be  smaller  than  the  target  label  error  from  single-look  identification. 


6.1  Physical  characterization  of  the  Score-Level  Fusion 
method 

In  an  attempt  to  understand  more  clearly  why  the  score-level  multi-look  sequence  produces 
significantly  better  identification  results  than  the  single-look  input,  it  is  helpful  to  examine  the 

mean  and  standard  deviation  (i.e.,  the  square  root  of  the  variance)  of  the  quantity  ( R, }  as  defined 
by  Equation  (14).  The  standard  deviation  is  defined  as, 


M 


n= 1 


il/2 


(16) 


where  A,  is  the  overall  mean  value  of  the  maximum  cross-correlation  response  R(x,  pt)  from  a 
large  set  of  test  samples  examined  for  target  t  before  partitioning  into  single-look  and  multi-look 
sequences;  i.e., 

(17) 

TV  is  a  large  integer.  For  a  k-multi-look  sequence,  M=(k  - 1 )  if  k  >1  and  M  =  1  if  k  -  1.  It  should 
be  noted  that  N  »  k;  that  is  to  say,  we  are  analyzing  the  statistics  of  multi-look  sequences  using  a 

large  set  of  samples.  Figures  10  and  1 1  show  the  graphs  of  the  mean  values  (circles)  for  ( R ,} 
between  a  test  target  and  each  of  the  reference  targets  stored  in  the  target  classifier  for  single-look 
and  for  a  10-look  sequence  respectively.  They  correspond  to  the  identification  cases  for  the 
square  and  diamond  data  points  shown  on  the  bottom  heavy  solid  curve  in  Figure  9,  respectively. 
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As  a  note,  the  sample  size  for  the  single-look  sequences  is  N=  25624,  and  the  sample  size  for  the 
10-look  sequences  is  N=  2562. 

The  mean  values  for  (R,)  in  Figures  10  and  1 1  are  plotted  as  a  function  of  the  target  types  stored 
in  the  classifier  for  all  multi-look  test  sequences  before  the  identification  of  the  test  targets  are 
determined  using  Equation(15).  The  standard  deviations  are  displayed  as  error  bars.  There  are  14 
graphs  displayed  in  each  of  Figure  10  and  Figure  11.  Each  graph  represents  one  of  the  14 
different  input  test  target  types  as  indicated  at  the  top  of  each  graph.  The  horizontal  axis  on  each 
graph  corresponds  to  the  1 1  reference  target  types  (see  figure  captions)  that  are  stored  in  the 
target  classifier.  Note  that  the  correct  reference  target  type  that  corresponds  to  the  test  target  in 
each  graph  is  indicated  by  an  arrow  in  each  graph. 

It  would  be  insightful  to  examine  the  mean  and  the  standard  deviation  of  the  (R,)  values  for  each 
reference  target  type  in  the  classifier  on  each  of  the  graphs  in  Figures  10  and  11.  This  effectively 
amounts  to  examining  and  comparing  the  behavior  of  the  (Rt)  values  by  monitoring  how  much 
the  standard  deviation  (error-bar)  changes  around  the  mean  (circle)  as  the  number  of  test  samples 
increases  from  a  single  look  to  10  looks  in  the  sequence.  Note  that  the  mean  values  for  (R,)  in 

Figures  10  and  11  are  the  same,  regardless  of  the  number  of  samples  in  the  multi-look  sequence. 
This  can  be  explained  by  the  fact  that  the  overall  mean  value  of  a  given  total  number  of  samples 
N  is  fixed,  regardless  of  how  this  number  N  is  partitioned  into  multi-look  sequences  of  different 
sample  sizes;  that  is  to  say,  whether  a  sequence  contains  1  sample  or  10  samples,  the  total  number 
of  test  samples  involved  is  the  same  and  hence  its  overall  mean  value  A  remains  unaltered. 
However,  the  telltale  sign,  that  the  sample  size  in  the  multi-look  sequence  plays  a  role  in  the 
identification  process,  is  the  reduction  of  the  size  of  the  error  bars  as  the  sample  size  of  the  multi¬ 
look  sequence  becomes  bigger.  This  can  be  seen  by  comparing  the  size  of  the  error  bars  of  the 
respective  graphs  between  single-look  and  10-look  as  shown  in  Figures  10  and  11,  respectively. 

The  reduction  in  the  standard  deviation  with  increasing  sample  size  in  the  multi-look  sequence 
offers  an  explanation  in  a  statistical  manner  of  the  observed  improvement  in  the  identification 
results.  As  a  reminder,  in  each  of  the  graphs  shown  in  Figures  10  and  11,  the  mean  values 
(circles)  provide  a  measure  of  similarity  between  the  test  signature  and  signatures  of  the  target 
types  stored  in  the  target  classifier  before  an  identification  is  declared.  Graphically,  the  larger  the 

mean  value  for  (R,) ,  the  more  probable  that  the  input  test  target  would  be  identified  as  that  target 

type.  It  may  be  easier  now  to  see  that  when  the  standard  deviation  around  the  mean  value  is  large, 
as  in  the  single-look  case  in  Figure  10,  there  is  a  bigger  chance  statistically  that  the  test  input  may 

not  be  correctly  identified  because  the  (Rt)  values  for  some  incorrect  reference  target  types  in  the 

classifier  may  have  a  larger  value  than  the  actual  correct  target  type  within  the  larger  error  margin 
as  indicated  by  the  error  bar.  As  the  number  of  test  samples  is  increased  to  10  in  the  multi-look 
sequence  (Figure  11),  the  error  bar  becomes  much  smaller.  It  can  be  seen  from  Figure  1 1  that  the 

(. R, )  value  corresponds  to  the  correct  target  type  is  more  likely  to  be  larger  than  those  that 

correspond  to  the  wrong  target  types  as  the  (R,)  values  are  approaching  the  mean  values  with  very 
small  errors;  hence  a  more  accurate  identification  can  be  made.  In  other  words,  it  is  much  more 
likely  that  the  reference  target  in  the  classifier  with  the  largest  mean  value  for  ( Rt }  is  the  correct 


DRDC  Ottawa  TM  2008-251 


21 


target  type  associated  with  the  input  test  target  in  the  10-multi-look  sequence.  This  is  evident  in 
Figure  1 1  where  the  correct  matches  are  indicated  by  the  arrows,  and  the  mean  value  indicated  by 
the  arrow  has  the  largest  value  in  each  of  the  graphs. 

In  summary,  when  the  number  of  samples  in  a  multi-look  sequence  is  small,  the  averaged 
correlation  values  (Rt) ,  between  the  test  input  and  the  target  classes  in  the  classifier,  can  vary 
quite  considerably;  i.e.,  having  a  large  error-bar  associated  with  the  mean  values.  Hence  the 
chance  of  the  correlation  values  from  the  wrong  target  type  having  a  greater  value  than  the 
correlation  value  from  the  correct  target  type  is  greater;  this  translate  to  a  lower  probability  of 
correct  identification.  As  the  number  of  test  samples  increases  in  the  multi-look  sequence,  the 
correlation  values  have  much  less  variability;  i.e.,  much  smaller  error-bars  associated  with  the 
mean  values.  Thus,  the  probability  of  making  a  correct  identification  is  much  higher.  The  mean 

and  standard  deviation  of  the  quantity  (Rt)  provide  a  qualitative  characterization  of  the 
identification  results.  In  essence,  the  correct  identification  rate  improves  statistically  with 
increasing  number  of  test  samples  in  the  multi-look  sequence  because  there  is  less  error 

associated  with  the  averaged  correlation  value  (Rt) . 

The  principle  of  averaging  that  provides  a  higher  correct  identification  rate  may  also  be  used  to 
explain  why  the  score-level  fusion  method  appears  to  be  independent  of  the  quality  of  the  data. 
Figure  12  shows  that  the  means  and  standard  deviations  for  the  10-look  case  using  the  best 
quality  signature  data;  i.e.,  the  “star”  data  point  on  the  top  solid  curve  in  Figure  9.  Comparing 
Figure  12  with  Figure  1 1,  it  can  be  seen  that  the  mean  value  on  the  column  indicated  by  the 
arrows  in  each  of  the  graphs  is  the  largest;  it  indicates  the  correct  match  between  the  input  test 
target  and  the  reference  target.  The  quality  of  the  data  is  reflected  in  the  magnitude  of  the 
correlation  values.  The  correlation  values  from  good  quality  data  (Figure  12)  are  larger  than  those 
from  poorer  quality  data  (Figure  11).  But  the  identification  decision  making  process  is  solely  a 
problem  of  finding  the  relative  maximum  correlation  value  (i.e.,  Equation  (15)).  Relative  score 
implies  the  absolute  magnitude  of  the  correlated  value  is  not  relevant.  Thus,  this  provides  an 
explanation  as  to  why  the  correct  identification  outcome  is  independent  of  the  data  quality. 

The  principle  of  averaging  is  a  well-tested  and  well-accepted  concept;  it  is  used  in  many  real- 
world  practical  applications.  Averaging  is  a  simple  procedure;  often,  simplicity  is  also  associated 
with  robustness.  Thus,  the  “statistics  of  average”  appears  to  provide  a  reasonable,  qualitative 
explanation  of  why  the  score-level  fusion  method  works  so  well. 
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Figure  10  Means  (circles)  and  standard  deviations  (error  bars)  for  single-look  using  the  lowest 
quality  data  set.  The  error  bars  have  a  magnitude  of  ±2  sigma.  The  horizontal  axis  on 
each  graph  represents  the  reference  target  types  that  are  stored  in  the  classifier:  1  = 
BMP2(c21),  2  =  172(132),  3  =  BRT70,  4  =  SLICY,  5  =  BTR60,  6  =  BRDM2,  7  =  ZSU, 
8  =  T62,  9  =  ZIL,  10=2S1,  11=D7.  The  arrows  indicate  the  correct  target  types  stored 
in  the  classifier  that  are  associated  with  the  test  input  target  types. 
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Figure  11  Means  (circles)  and  standard  deviations  (error  bars)  for  10-look  using  the  lowest 

quality  data  set.  The  error  bars  have  a  magnitude  of  ±2  sigma.  The  horizontal  axis  on 
each  graph  represents  the  reference  target  types  that  are  stored  in  the  classifier:  1  = 
BMP2(c21 ),  2  =  172(132),  3  =  BRT70,  4  =  SL1CY,  5  =  BTR60,  6  =  BRDM2,  7  = 
ZSU,  8  =  T62,  9  =  ZIL,  10—2S1,  11=D7.  The  arrows  indicate  the  correct  target  types 
stored  in  the  classifier  that  are  associated  with  the  test  input  target  types. 
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Figure  12  Means  (circles)  and  standard  deviations  (error  bars)  for  10-look  using  the  best  quality 
data  set.  The  error  bars  have  a  magnitude  of  ±2  sigma.  The  horizontal  axis  on  each 
graph  represents  the  reference  target  types  that  are  stored  in  the  classifier:  1  = 
BMP2(c21),  2  =  T72(132),  3  =  BRT70,  4  =  SLICY,  5  =  BTR60,  6  =  BRDM2,  7  =  ZSU, 
8  =  T62,  9  =  ZIL,  10—2S1,  11=D7.  The  arrows  indicate  the  correct  target  types  stored 
in  the  classifier  that  are  associated  with  the  test  input  target  types. 
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6.2  Empirical  exponential  fit  to  the  observed  results 

The  improvement  in  the  correct  identification  rates  as  seen  in  Figure  9  using  score-level  fusion 
(solid  curves)  appears  to  be  very  rapid  with  an  increase  in  the  samples  of  the  multi-look  sequence. 
The  rate  of  increase  is  observed  to  be  similar  and  is  independent  of  the  data  quality.  In  an  attempt 
to  quantify  the  characteristics  of  the  improvement  in  the  correct  identification  rate,  the  standard 

deviation  (error-bar)  for  (Rt)  is  again  examined  to  provide  some  extra  insights;  it  is  seen  from  the 

discussion  in  Section  6.1  that  the  correct  identification  rate  increases  as  the  size  of  the  standard 
deviation  decreases.  Figure  13  illustrates  the  size  of  the  error-bar  as  a  function  of  the  number  of 
multi-look  samples  for  each  of  the  4  different  data  quality  cases.  It  is  seen  that  the  error-bar  sizes 
are  about  the  same  for  all  4  data  quality  cases  and  are  found  to  decrease  rapidly  with  an  increase 
in  multi-look  samples. 

Another  perspective  on  the  error-bar  behaviour  is  given  by  normalizing  the  error-bars,  i.e., 


■*s,N 


(18) 


where  aSiN  is  the  A-sample  multi-look  error-bar  size,  asj  is  the  single-look  error-bar  size  and  s  is 
an  index  indicating  the  state  of  the  data  quality,  e.g.,  s  =  1,  2,  3,  4  indexing  the  4  different  states 
of  data  quality.  The  normalized  error-bar  size  is  illustrated  in  Figure  14a.  It  can  be  seen  that  the 
normalized  error-bars  are  essential  the  same  in  size  among  the  4  data  quality  cases. 

An  empirical  fitting  equation  to  the  observed  identification  results  in  Figure  9  using  the 
normalized  error  bar  as  a  parameter  can  be  expressed  as, 

fs,N  =  1  -  exP  (-9A  +  °-3)  )  (19) 


where  fS:N  is  the  fitted  correct  identification  rate,  N  is  the  number  of  samples  in  the  multi-look 
sequence,  gjvis  the  fitting  function  for  the  normalized  error-bar  size  as  shown  in  Figure  14b  and 
qs  i  is  a  fitting  parameter  such  that  the  fitted  single -look  correct  identification  rate/, ,  is  matched 
to  the  measured  single-look  correct  identification  rate  as  an  initial  condition  that  satisfies  the 
empirical  fit  in  Equation  ( 19);  in  other  words, 

(20) 

where  Pid  ( s ,  1 )  is  the  single-look  value  of  the  correct  identification  rate  from  measured  data  in 
Figure  9.  This  permits  a  common  starting  point  for  the  fitting  equation  (Equation  (19))  to  be 
compared  with  the  results  from  measured  data;  the  comparison  is  shown  in  Figure  15a.  It  can  be 
seen  that  the  computed  empirical  fits  (dashed  curves)  are  in  very  close  agreement  with  the 
measured  data  (solid  curves).  Note  that  in  Equation  (19),  there  is  a  fudge  factor,  0.3  added  to  the 
normalized  error-bar  size  variable,  sN  .  The  purpose  of  this  fudge  factor  is  to  provide  as  good  a  fit 
as  possible  between  the  empirical  fit  and  the  measured  data.  The  fudge  factor  does  not  have  any 
physical  interpretation  or  significance.  The  objective  of  the  empirical  fit  given  by  Equation  (19) 
is  to  provide  a  semi-quantitative  illustration,  showing  that  the  score-level  fusion  method  has  an 
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exponential  increase  in  the  identification  performance.  If  the  fudge  factor,  0.3  is  ignored, 

Equation  ( 19)  can  be  rewritten  as, 

fs,N=1-eM~Cls,l(7s,NN)  (21) 

Since  qsj  is  just  a  proportional  constant,  the  increase  in  the  modelled  correct  identification  rate  in 
this  case  is  proportional  to  the  product  os?nN  in  an  exponential  manner.  Figure  15b  shows  the 
empirical  fits  given  by  Equation  (21)  (dashed  curves),  and  a  comparison  with  the  measured 
results  (solid  curves)  is  made.  Having  the  fudge  factor  set  to  0,  the  empirical  fits  are  increasing 
more  slowly  than  the  measured  data.  In  either  case,  an  exponential  rise  in  the  corrected 
identification  rate  with  an  increase  in  the  multi-look  samples  is  evident  in  the  score-level  fusion 
method  regardless  of  data  quality. 

Thus,  the  score-level  fusion  method  demonstrates  that  quantity  can  replace  quality  of  the  data  to 
achieve  good  identification  accuracy.  However,  it  should  be  emphasized  that  quantity  can  not  be 
expected  to  replace  quality  in  an  arbitrary  manner.  In  other  words,  one  can  not  arbitrarily  apply 
quantity  to  replace  quality  and  expect  good  identification  automatically.  There  are  two  basic 
requirements  that  must  be  met  before  the  score-level  fusion  identification  can  be  useful.  Firstly, 
even  though  the  quality  of  the  target  signatures  may  be  degraded  by  whatever  conditions 
presented  during  data  acquisition,  they  must  possess  a  general  resemblance  that  is  consistent  with 
the  reference  signatures  deployed  in  the  classifier’s  database  of  the  same  target  types.  Secondly, 
the  classifier  should  have  a  robust  and  reliable  algorithm  that  could  give  consistently  a  higher 
probability  in  matching  the  input  signature  to  the  library  signature  of  the  same  target  type  than 
between  dissimilar  types.  In  other  words,  the  use  of  the  score-level  fusion  multi-look  procedure 
may  reduce  the  statistical  fluctuation  in  the  target  matching  process  that  causes  identification 
errors.  But  the  intrinsic  ability  of  the  classifier  algorithm  to  provide  correct  matches  of  similar 
looking  signatures  is  essential  and  is  still  central  to  the  performance  of  a  good  classifier.  These 
two  basic  requirements  can  not  be  expected  to  be  replaced  by  multi-look  procedure  alone  in  an 
arbitrary  manner.  As  discussed  in  Section  6.1,  correct  identification  is  achieved  only  if  a  classifier 
can  reliably  come  up  with  the  largest  matching  probability  (i.e.,  correlation  value)  between  two 
signatures  of  the  same  target  type.  Reasonable  datasets,  albeit  large  signature  variability  and  a 
robust  classifier,  are  still  essential  ingredients  for  accurate  identification. 
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Figure  13  Error  size  (standard  deviation)  as  a  function  of  the  number  of  multi-look  samples  for  4 
different  data  quality  cases. 


number  of  looks 


b 


Figure  14  Normalized  error  size  as  a  function  of  the  number  of  multi-look  samples,  a)  all  4  data 
quality  cases,  b)  fitted  normalized  error  size  En,  an  average  of  the  4  data  quality 
cases  as  shown  in  a. 
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probability  of  correct  ID 


Figure  15  A  comparison  of  the  identification  results  between  the  empirical  fit  and  the  measured 
data,  a)  fudge  factor  set  at  0.3,  green  solid  curves  =  measured  data,  black  dashed 
curves  =  empirical  fit  from  equation  (18);  b)  fudge  factor  set  at  0.0,  green  solid 
curves  =  measured  data,  black  dashed  curves  =  empirical  fit  from  Equation  (20). 
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7  Conclusions 


To  remedy  the  inadequate  performance  of  single-look  identification,  the  score -level  fusion 
method  is  investigated  as  a  multi-look  sampling  procedure  to  provide  highly  accurate  target 
identification  performance.  The  concept  of  score-level  fusion  has  been  known  for  some  time  and 
the  method  has  been  deployed  in  multi-modal  biometric  identification,  fusing  different  biometric 
information  together  to  obtain  more  accurate  identification.  What  is  new  and  original  in  this  study 
is  the  demonstration  of  significant  improvement  in  the  target  identification  accuracy  using 
multiple  samples  of  marginal  quality  signatures.  The  problem  of  improving  accuracy  using 
multiple  samples  of  poor  quality  data  has  largely  been  ignored  in  the  pattern  recognition 
community.  The  study  conducted  in  this  work  has  presented  a  new  and  innovative  perspective  on 
target  identification.  A  desirable  characteristic  has  been  identified  in  the  score-level  fusion 
process  that  can  provide  highly  accurate  identification  using  marginal  quality  target  signatures. 

The  superior  performance  of  the  score-level  fusion  method  is  highlighted  in  a  comparative 
analysis  with  the  majority-vote  method  using  experimental  target  signature  data  of  varying 
qualities.  Majority-vote  is  a  well-established  multi-look  procedure.  It  is  shown  that  the 
performance  of  the  majority-vote  method  is  dependent  on  the  target  data’s  quality.  Any  increase 
in  the  identification  performance  is  insignificant  when  the  target  signature  quality  is  marginal. 
When  the  target  signatures  are  particularly  poor  in  quality,  the  majority-vote  method  can  not  be 
used  to  improve  on  the  identification  performance  at  all.  But  on  the  other  hand,  the  score-level 
fusion  method  offers  very  significant  performance  improvement,  even  working  with  poor  quality 
data.  An  example  is  shown  in  which  the  correct  identification  rate  improves  from  37%  in  the 
single-look  mode  to  over  97%  in  the  multi-look  mode.  This  is  illustrated  in  Figure  9  (heavy  solid 
curve).  Moreover,  the  increase  in  performance  is  exponential  as  a  function  of  the  number  of 
samples  used  in  the  multi-look  sequence.  This  suggests  that  the  score-level  fusion  method  can 
provide  an  efficient  means  to  significantly  improve  the  identification  performance,  even  when 
poor  quality  data  are  used. 

The  “quantity  over  quality”  approach  to  exploiting  data  could  have  a  notable  impact  in  target 
identification  applications.  With  the  advent  of  sensor  technology,  large  quantities  of  target  data  of 
marginal  quality  can  be  made  available  routinely.  Whereas,  collecting  quality  data  still  presents  a 
technical  challenge.  Thus,  the  significant  improvement  in  identification  performance  and  the 
ability  to  maximize  the  exploitation  of  a  large  amount  of  data  offer  potential  niche  applications 
such  as  Aided  Target  Recognition,  radar  emitter  identification  [30]  and  human  biometric 
recognition  using  the  score-level  fusion  method. 

A  qualitative  characterization  of  the  score -level  fusion  method  based  on  the  statistics  of  average 
is  given  to  explain  the  excellent  identification  results  obtained  in  the  analysis.  There  is  very  little 
analytical  work  on  the  score-level  fusion  method  being  reported  in  the  literature.  Given  the 
positive  results  obtained  in  this  study,  it  would  be  useful  to  conduct  further  study  on  the  basics  of 
this  process  to  gain  better  insights.  This  presents  a  challenge  and  an  opportunity  to  put  the  score- 
level  fusion  method  in  a  more  rigorous  fundamental  framework.  A  good  conceptual 
understanding  of  the  fundamental  framework  could  have  significant  impact  in  opening  up  new 
applications  to  pattern  recognition  and  identification  problems,  particularly  in  the  Defence  and 
Security  S&T  field. 
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quantity  can  be  used  to  replace  quality  of  the  data  to  improve  target  identification  accuracy.  This 
has  an  interesting  practical  implication.  With  the  advent  of  sensor  technologies,  large  quantity 
of  data  of  marginal  quality  can  be  captured  routinely.  This  “quantity  over  quality”  approach 
could  maximize  the  exploitation  of  available  data  to  provide  reliable  and  robust  identification. 
Thus,  the  score-level  fusion  method  could  have  the  potential  of  being  developed  into  a 
dismptive  technology  for  target  identification  and  pattern  recognition  problems. 


1 4.  KEYWORDS,  DESCRIPTORS  or  IDENTIFIERS  (Technically  meaningful  terms  or  short  phrases  that  characterize  a  document  and  could  be 
helpful  in  cataloguing  the  document.  They  should  be  selected  so  that  no  security  classification  is  required.  Identifiers,  such  as  equipment  model 
designation,  trade  name,  military  project  code  name,  geographic  location  may  also  be  included.  If  possible  keywords  should  be  selected  from  a 
published  thesaurus,  e.g.  Thesaurus  of  Engineering  and  Scientific  Terms  (TEST)  and  that  thesaurus  identified.  If  it  is  not  possible  to  select 
indexing  terms  which  are  Unclassified,  the  classification  of  each  should  be  indicated  as  with  the  title.) 

multi-look  sampling,  score-level  fusion,  target  identification,  signature  quality,  pattern 
recognition. 
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